1. Field of the Invention
The invention relates to how data collected in field locations, often geographically diverse, is verified as being accurate. The comparison is made between data that have been collected, processed, and stored in a centralized location, along with any corrections or updates made since the data were first recorded, against original source data in the field. These steps are needed to verify the accuracy of data in the centralized database. Data refers to any information, including measurements, recordings, verbal responses, images (such as x-rays or CT scans), laboratory data, output from analyzers, or any other source, including electronic, paper, or other means.
Increasingly, clinical research is also under constraints to improve the ability to manage complex clinical trials, which are generally geographically diverse. Doing so requires continuous measurement of numerous performance indices, an easy reporting mechanism, and the ability to intervene or otherwise change processes, practices, or other elements to improve performance.
2. Description of the Related Art
Studies performed as part of evaluation of pharmaceutical products rely on different forms of data collected in the field, all of which must be accurately handled during collection, processing, and updating. It is therefore highly desirable, and may be required, that each piece of data that serves as the basis for analyzing the results of research, must accurately reflect field data. In pharmaceutical research, the first time a piece of data is recorded, it is considered “source,” or the correct value. Any research, whether pharmaceutical or not, must have analyses based on data that are a true and accurate representation of such source data, and any group or individual conducting research must assure that data have been recorded and processed accurately and, when changes are made between the time data are first recorded and entered in the final database, that each step does not introduce errors. This is generally accomplished by comparing data in the final database against “source” data, which may be defined as the location in which a piece of data was first recorded.
Pharmaceutical research is generally conducted in hospitals, clinical, physician offices, and other medical locations in which source data may reside in patient's charts, which may be electronic or paper. Current systems of data entry fall into two general categories: most (about 50 to 60% of current clinical studies) involve recording a value on a paper Case Report Form (“CRF”), after which it is entered by a data entry clerk or the like, who types each value into an electronic system. A second verification entry (“double key entry”) is then performed as a quality check. The other means of data entry, currently employed by approximately 40 to 50% of clinical trials, involves web-based Electronic Data Collection. This generally involves using a Worksheet onto which data are copied from source data (or “Source”), and from which the data then are keyed into an electronic system at the site.
In addition to the integrity of clinical data, data about the data themselves (meta-data) are useful and highly desirable means of measuring the practical aspects of how studies are conducted. These meta-data include site performance metrics such as error rates on data submitted, enrollment rates, and other elements key to the timely performance of each site and for the study overall. These data may be analyzed in different fashions, including point estimates, trends over time, or relative to other sites or performance measures and are described in U.S. Ser. No. 60/926,577.
Data are currently processed by comparing incoming data against as many as several hundred validation rules. These types of checks generally fall into three groups: (1) Range checks, which assure that a value for a data field falls within expected parameters and that the data are of the proper type (e.g., generally alpha or numeric); (2) Consistency, where an answer to one question may limit the responses to another question (for example, if subject is male, number of pregnancies should equal zero or Not Applicable); and (3) Trend information, where parameters may be specified for rate of change of certain variables (for example, a hemoglobin value may be consistent for the first four study visits but then drops precipitously, or height may be recorded as significantly lower than at previous visits). Being able to assess this information very quickly and to provide feedback to the sites that collected the data is very important to being able to assure that similar errors are not repeated.
Prior systems have utilized a web-based means of collecting data, transmitting this over computer networks or telephone lines, and putting data in a database. See, for example, U.S. Pat. No. 6,496,827, which discloses a system wherein data are input at a computer at the time of collection, for transmission over the internet to a central data storage site or database. The data are input in real time, via a graphical user interface that also provides means for rudimentary validation of the data. Further validation of the data occurs at the central site, via comparison with other data already in the database. However, systems such as those disclosed in U.S. Pat. No. 6,496,827 are limited by a lack of flexibility in how data are collected; require separate systems to perform data validation; and do not track study performance metrics.
The ability to analyze clinical data (and corresponding meta-data) is important and increasingly time sensitive, in part because the ability to produce more rapid decisions is based on the rapid availability of accurate data. This capability lies at the heart of adaptive clinical research, techniques and processes by which data and meta-data can be continuously reviewed and incorporated into changes in how studies are conducted. (In the context of the present invention, “adaptive” means that the course of such clinical investigations could be altered based on experience as a study progresses. Close monitoring of performance metrics allows early identification of weaknesses and allows these to be addressed, providing a more effective management system.) These changes may be, but are not limited to, study design, such as number or allocation of subjects, or to operational elements such as how to track performance of study sites or interviewers, subject recruitment strategies, and allocation of resources such as management.
Currently verification of site data is performed in one of two ways. The first is for data to remain at the site until a field monitor (also known as a “Clinical Research Associate” or “CRA”) visits the site, at which time data are manually reviewed, errors that are detected are corrected on the scene, and data are then brought to a centralized location for data entry and computerized checking for range (allowable values), consistency (if one answer constrains answers that may appear elsewhere), and possibly other checks.
This method, however, has serious drawbacks in that such data and the corresponding meta-data typically are not available for weeks to months after they are actually generated, presenting serious obstacles to being able to measure elements such as site performance with sufficient time to allow effective management and seriously hampering, to the point of effectively precluding, the application of adaptive techniques to either strategic study elements or to the effective management of the study itself. This also creates considerably more work for clinical sites, since discrepancies are identified a long period of time after they are actually made, and the same mistakes may continue to occur in the interim. In addition, any errors that are identified require substantially more effort to go back and correct, since they occurred weeks ago, and paperwork already has been filed.
The second possibility is that data are submitted after they are generated, but before a monitoring visit has occurred. Under this scenario, data are entered and validation checks applied, usually at a centralized location. At some time after the data are received and validated (usually several weeks, but sometimes as long as several months), the monitor returns to the site to check each value in the centralized database against source data.
In either scenario, preparation for such a field visit occurs by printing out the database values, and printing a separate list of changes that occurred during the time between when a data point was first entered and the entry of its final value in the database. In practical terms, this often amounts to printing out several hundred sheets of paper, transporting these printouts to a clinic or other location where patients are seen, and comparing each value on the paper with a separate paper record of changes against the original (“source”) data. This process is laborious, time-consuming, and error prone, since large stacks of paper are difficult to handle, sheets can be lost, and errors noted with follow-up flags (e.g., “Post-It Notes” or “stickies”) can be lost because they are hidden from sight, fall off, or shift. Errors of omission can occur under such circumstances and often go undetected, since there is no backup mechanism whereby such errors would be suspected or detected. Each discrepancy that is noted results in an individually-written note (a “query”) that must be transmitted to the site and also tracked as a change so that an “audit trail” is maintained. (An audit trail typically comprises a list of entries, each containing an old value, a new value, an identification of the person who changed the value, and the reason for the change.) The queries are generally returned to the site by fax. The entire process is time-consuming and similarly prone to error.
In either of these scenarios, a considerable amount of time is also required to prepare for a site visit and to enter the results of site work following the visit. In the first case, the field monitor has no guidance about where or what kind of errors might exist and may review several thousand fields during a typical monitoring visit. Remaining vigilant for errors during such a review is tedious and difficult, and field monitors with differing levels of experience may produce dramatically different results. Preparing for a field monitoring visit generally requires a day of preparation that includes printing out several hundred sheets of paper to prepare for a two-day site monitoring visit.
Thus, it would be desirable to be able to verify field data more promptly, and in a less labor-intensive and error-prone manner.